Image processing apparatus and method for controlling the same

ABSTRACT

An image processing apparatus generates an image by adding a plurality of different film-tone image effects to an input image. When the plurality of different film-tone image effects is added to an input image, if the effects include image slurring or blinking, an addition order is determined so that such effects can be added after the other effects.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus thatprovides a film-tone effect to digital image data, and a method forcontrolling the same.

2. Description of the Related Art

Recently, there has been a method for carrying out digital photographingwith various expressions by adding an effect as if film photographing isexecuted, to a captured image as one of image expression of a digitalcamera. Japanese Patent Application Laid-Open No. 2001-346218 discussesa method for carrying out, in order to achieve color reproduction andgradation obtainable by film photographing, predetermined conversion ofa digital image to bring it close to a film photography. Japanese PatentApplication Laid-Open No. 11-18005 discusses a method for sequentiallyperforming the processes by a user's operation in a case where aplurality of film-tone effects is exerted. However, the order of addinga plurality of film-tone image effects to the image data has only beenmanually determined by the user's operation.

SUMMARY OF THE INVENTION

The present invention is directed to an image processing apparatus thattakes into account an order of adding image effects when processing foradding a plurality of different film-tone image effects is carried outfor image data, and, a method for controlling the same.

According to an aspect of the present invention, an image processingapparatus includes: an acquisition unit configured to sequentiallyacquire image data; a first processing unit configured to carry outimage distortion processing on the image data; a second processing unitconfigured to add noise to the image data processed by the firstprocessing unit; and a third processing unit configured to add slurringto the image data processed by the second processing unit andsequentially output.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating an imaging apparatus according toa first exemplary embodiment.

FIG. 2 is a block diagram illustrating an imaging apparatus according toa second exemplary embodiment.

FIG. 3 illustrates a process starting from image capturing toreproduction in the case of a film camera.

FIG. 4 illustrates a phenomenon and a film-tone effect in the case ofthe film camera.

FIG. 5 illustrates an order of adding effects according to the firstexemplary embodiment.

FIG. 6 illustrates priority of each effect according to the firstexemplary embodiment.

FIG. 7 illustrates an order of adding effects according to a secondexemplary embodiment.

FIGS. 8A to 8D illustrate priority of each effect according to thesecond exemplary embodiment.

FIG. 9 is a flowchart illustrating processing for determining the orderof adding effects according to the first exemplary embodiment.

FIGS. 10A and 10B include a flowchart illustrating processing fordetermining the order of adding effects according to the secondexemplary embodiment.

FIGS. 11A to 11C illustrate effects of distortions according to thepresent invention.

FIG. 12 illustrates an effect of grain noise according to the presentinvention.

FIG. 13 illustrates an effect of scratch noise according to the presentinvention.

FIG. 14 illustrates an effect of color fading according to the presentinvention.

FIG. 15 illustrates an effect of vertical slurring according to thepresent invention.

FIGS. 16 illustrates an effect of blinking according to the presentinvention.

FIG. 17 illustrates an effect selection screen according to the presentinvention.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

In a digital camera, when adding a plurality of types of film-tone imageeffects to an image, an order of adding the effects must be taken intoaccount. An example where different effects, namely, a grain noiseeffect for reproducing granular noise generated due to filmcharacteristics or during development and a color reproduction effectfor reproducing color fading caused by a secular change of film, will bedescribed.

FIG. 3 illustrates a process starting from moving image capturing toreproduction in a film camera. FIG. 4 illustrates realization ofprincipal effects caused by a phenomenon occurring during each processillustrated in FIG. 3, by image processing. The process from filmphotographing to reproduction is divided into four, namely, (1)photographing, (2) development/editing, (3) storage, and (4)reproduction. The film camera passes through the process from theupstream photographing to the downstream reproduction.

In the first process (the photographing), an image is captured on a filmby using a film camera 301. In this process, a photosensitive filmsurface is exposed to incident light, and an image is formed thereon. Adifference in optical characteristics of a lens installed in the filmcamera 301 or chemical characteristics of the film causes a principalphenomenon. An example of an effect caused by such a phenomenon isdistortion aberration based on the difference in optical characteristicsof the lens.

In the second process (the development/editing), the film image isdeveloped and edited. In this process, a difference in development timeor method during film image development or damage of a film surface 302caused by a development work mistake causes a principal phenomenon. Anexample of an effect generated by a difference in development method isgranular noise. An example of an effect generated by the damage of thefilm surface 302 is scratch noise.

In the third process (the storage), the developed and edited film 303 isstored. In this process, alteration of the film 303 that has occurreddue to deterioration with time caused by a chemical change causes aprincipal phenomenon. An example of an effect generated by thealteration of the film is color fading.

In the fourth process (the reproduction), the film is set on a projector304 to show the captured images. In this process, rotational unevennessof a reel on which the film is wound or fluctuation in brightness of alight source of the projector 304 causes a principal phenomenon. Anexample of an effect generated by the rotational unevenness of the reelis vertical slurring. An example of an effect generated by brightnessunevenness of the light source is blinking.

Thus, the phenomena of granular noise and color fading occur in thisorder in the film photographing. It is therefore desirable to add, whenadding a plurality of film-tone image effects, the image effects in thesame order as that of the phenomena appearing during the filmphotographing.

Thus, according to the present exemplary embodiment, when carrying outprocessing for adding image effects to acquired image data, the imageeffects are added in the same order as that of the phenomena appearingduring the film photographing.

FIG. 1 is a block diagram illustrating a digital video camera as animage processing apparatus according to a first exemplary embodiment.The digital video camera includes: an optical unit 100 that includes alens, a diaphragm, and an actuator for driving these components; animage capturing unit 101 that processes signals read from an imagesensor to sequentially acquire and output image data; a recording unit102 that records a video signal; and a key switch 103 that includes agroup of switches for enabling the user to select and determinefilm-tone image effects.

A group of lenses 104 and 105 forms an object image. A diaphragm 107adjusts the amount of incident light. A driving control unit 106includes an actuator for driving the lens 104 and the diaphragm 107 tohave instructed predetermined values. An image sensor 108photoelectrically converts incident light which passes through theoptical unit 100 to form an image. The photoelectrically convertedsignal is input to a camera signal processing unit 109.

The camera signal processing unit 109 carries out various imageprocesses for the signal photoelectrically converted into a videosignal. An image effect processing unit 113 adds various film-tone imageeffects to the video signal output from the camera signal processingunit 109. The video signal output from the image effect processing unit113, on which a predetermined user interface screen (hereinafter, UIscreen) is superimposed, is displayed on a display unit 134. An encoder132 codes the video signal output from the image effect processing unit113 in a predetermined recording format to write and store it in arecording medium 133.

Next, signal processing carried out by the camera signal processing unit109 will be described. The signal input to the camera signal processingunit 109 is separated into a luminance component and a color component.Various signal processes such as gamma processing, color matrixprocessing, and contour enhancement processing are performed on theseparated luminance component and the color component. A frame memory115 stores an image processed by the image effect processing unit 113.The image is written or read at predetermined timing.

The image effect processing unit 113 adds a plurality of differentfilm-tone image effects to the input image. The effects to be added are“distortion” for adding a distortion, “grain noise” for adding granularnoise, “scratch noise” for adding vertical noise, “color fading” forreducing saturation, “blinking” for adding fluctuation of a luminancelevel which changes with time, and “vertical slurring” for addingvertical slurring. The effects added by the image effect processing unit113 realize by image processing the same effects as those of thephenomena appearing in the process from recording (hereinafter,photographing) to reproduction (hereinafter, reproduction) in movingimage photographing using a film as a recording medium.

This provides the effects of pseudo film photographing and reproductioneven for a digital image captured by the image sensor according to thepresent exemplary embodiment. To realize distortion aberration at theimage effect processing unit 113, projective transformation processingis carried out for the image. In order to cause the image effectprocessing unit 113 to realize granular noise generated on the filmsurface, a grain noise image prepared beforehand is synthesized with theinput image.

Similarly, in order to cause the image effect processing unit 113 torealize scratch noise due to damaging of the film surface, a scratchnoise image prepared beforehand is synthesized with the input image. Inorder to realize color fading due to deterioration with time by theimage effect processing unit 113, a saturation level of the image isreduced.

Rotational unevenness of the reel appears as, for example, a verticalchange of a scene being reproduced. Accordingly, the rotationalunevenness can be realized by the image effect processing unit 113adding a vertical slurring effect to the image. Fluctuation inbrightness of the light source can be realized by adding a blinkingeffect to provide fluctuation of a luminance level which changes withtime, to the image.

A system controller 114 controls the image sensor 108, the camera signalprocessing unit 109, and the image effect processing unit 113. Thesystem controller 114 designates a signal storage period or readingtiming to the image sensor 108. The system controller 114 sets, for thecamera image processing unit 109, parameters necessary for image qualitysetting by various signal processes. The system controller 114 acquiresevaluation values necessary for exposure control, focus control, andwhite balance control from the camera image processing unit 109.

The system controller 114 detects control positions of the lens 104 andthe diaphragm 107, determines, based on the evaluation values acquiredfrom the camera image processing unit 109, control values so that thecontrol positions can be placed at desired positions, and thendesignates the control values to the actuator 106. The system controller114 instructs each image effect processing unit 113 to determine anexecution order of effects and execute various setting operations.

Next, the image processing of adding the film-tone image effects at theimage effect processing unit 113 will be described in detail. FIGS. 11Ato 11C illustrate projective transformation processing (distortionprocessing, first processing) for realizing “distortion” according tothe present exemplary embodiment.

In the projective transformation processing, the input captured image isdivided into a predetermined number of regions, and each intersectionpoint of the divided region is moved to predetermined coordinates,thereby realizing deformation of the input image. A minimum unit of thedivided region is one pixel constituting an image, and a minimum unit ofcoordinate movement is also one pixel. An absolute moving amount of eachintersection point is point-symmetrical with respect to an image center,and a concentrically varying distortion effect can be realized.

Distortion aberrations are classified into a barrel type and a bobbintype. These two types of distortion aberrations can be realized bychanging movement characteristics of the intersection point. The barreltype can be realized by moving outward a coordinate position of theintersection point from the image center. By an effect of thebarrel-type distortion, an intersection point 1101 before deformationillustrated in FIG. 11A moves to an intersection point 1102 afterdeformation illustrated in FIG. 11B.

On the other hand, the bobbin-type distortion can be realized bydeformation toward the image center. By an effect of the bobbin-typedistortion, the intersection point 1101 before the deformationillustrated in FIG. 11A moves to an intersection point 1102 afterdeformation illustrated in FIG. 11C. A difference in effect between thebarrel type and the bobbin type is provided as a type of a distortioneffect to the projective transformation processing unit.

A deformation amount calculation unit 116 illustrated in FIG. 1calculates a deformation amount added to the input image. Thedeformation amount calculation unit 116 determines a moving amount of anintersection point of each region when the image input from the framememory 115 is divided into predetermined regions. A deformationprocessing unit 117 carries out projective transformation processing forthe input image according to the deformation amount determined by thedeformation amount calculation unit 116.

The deformation amount calculation unit 116 can acquire intersectionpoint moving amounts of a plurality of patterns based on parametersacquired from a plurality of deformation data stored in a memory 135.Thus, an image where different distortion characteristics are added tothe same input image can be acquired by selecting a deformation amountdata parameter. In other words, an effect similar to an image capturedby a lens of optical characteristics of distortion types at variouslevels or distortions of a barrel type, and a bobbin type can beprovided.

FIG. 12 illustrates a relationship between noise data constituting thegrain noise and clipped noise data. A grain noise memory 118 storestwo-dimensional noise data 1201 as grain noise. A grain clippingprocessing unit 119 clips grain noise data 1202 of a predetermined sizeand data from the grain noise memory 118. A grain resizing processingunit 120 resizes the clipped grain noise data 1202 to grain noise data1203 of a size required for synthesis with the input image. A grainsynthesis processing unit 121 reads a captured image stored in the framememory 115, and synthesizes the image with the grain noise data 1203 ata predetermined synthesis ratio to store it in the frame memory 115.

FIG. 13 illustrates a relationship between noise data including scratchnoise of a plurality of patterns and data clipped therefrom. In scratchnoise data 1301, one pixel in a horizontal direction is a minimum unit,a scratch flaw in a vertical direction is stored, and its level isindicated by a random number. A scratch level changes in the verticaldirection with a plurality of pixels set as a minimum unit. Accordingly,a depth or a thickness of the scratch noise changes in the verticaldirection, and a “blur” of the flaw is represented. Various randomnumbers such as that of Gaussian distribution can be employed. However,they are not limited to any specific types of random numbers.

In noise synthesis, noise data 1302 is clipped from the scratch noise1301, and resized to a predetermined image size to generate noise data1304 to be pasted. Then, depending on a pasting position and pastingduration at that position in the previous pasted noise data 1304, apasting position of the current noise data 1304 to be pasted isdetermined to synthesize the data with the captured image.

A scratch noise memory 122 illustrated in FIG. 1 stores the scratchnoise data 1301 of the plurality of patterns. The scratch noise data1301 is read from the scratch noise memory 122. A scratch clippingprocessing unit 123 clips scratch noise data 1302 at a designatedposition and of a designated size from the scratch noise data 1301. Ascratch resiting processing unit 124 resizes the clipped noise data 1302to scratch noise data 1304 of a size required for synthesis with thecaptured image 103 stored in the frame memory 115. A scratch synthesisprocessing unit 125 reads the captured image stored in the frame memory115, and synthesizes the image with the resized noise data 1304 at apredetermined synthesis ratio to store it in the frame memory 115.

FIG. 14 illustrates correction processing of a color difference signalto realize color fading according to the present exemplary embodiment.In the color difference correction processing, saturation of a videosignal is reduced by principally changing a color difference componentof an output image based on predetermined input/output characteristicsfor the input captured image, thereby realizing color fading. Forexample, when the input image is in a YUV format, correction processingis carried out for data of a U signal and a V signal that are colordifference components.

Correction processing can also be executed for a Y signal whenappropriate. Thus, as a result, only intensity of a color component ischanged while maintaining a luminance signal of the image. A minimumunit of a change of the color difference signal is minimum resolution ofa signal of the output image.

The memory 135 illustrated in FIG. 1 stores a plurality of colordifference characteristic data for determining input/outputcharacteristics of the color difference of the video signal. Forexample, in the case where characteristics 1401, 1402, and 1403 arerepresented by a linear function, parameter data includes inclinationand intercept of a straight line. The characteristics 1401 indicatenormal characteristics showing no color fading. The characteristics 1402indicate characteristics of color fading, i.e., lighter color, comparedwith the characteristics 1401. The characteristics 1403 indicatecharacteristics of much lighter colors than the characteristics 1402.

Characteristic data is determined according to a level of a secularchange, for example, the number of years elapsed after film development.The determined parameter data is transmitted to a color differencecorrection processing unit 126. The color difference correctionprocessing unit 126 corrects the input/output characteristics of thecolor difference for the captured image read from the frame memory 115according to the parameter data indicating the determinedcharacteristics, and outputs the corrected captured image to the framememory 115. Further, as described above, correction processing of, forexample, a gain, can be carried out for the luminance signal.

FIG. 15 illustrates a data structure in the frame memory 115 that storesthe image captured by the image sensor, and an image displayed when thedata is clipped at an arbitrary position from the frame memory 115. Thedata in the frame memory is sequentially updated. Data for a differentpurpose is stored in a region preceding or following the captured imageto be subjected to vertical slurring processing. However, this data canbe regarded as noise data since it has no relation to the verticalslurring processing.

When the captured image is selected at a predetermined clipping startposition within a predetermined clipping range, the image is output as adisplay image 1500. At this time, by determining the clipping startposition based on a random number, the image can be clipped from arandomly determined position. As a result, the captured image to whichan effect of vertical slurring has been added is output as a displayimage 1501 or 1502.

The clipping start position is determined with one pixel of the image inthe vertical direction, namely, one line is set as a minimum unit.Various random numbers such as that of Gaussian distribution can beemployed, however, they are not limited to any specific types of randomnumbers.

Vertical slurring beyond a certain amount is prevented by placing anupper limit on an offset amount (slurring amount) from a referenceposition to the clipping start position. As for the vertical slurring, astate of no slurring is set as the reference position. A sum of twovertical slurring amounts determined by different cycles is set as theclipping start position. This enables representation of verticalslurring configured of a combination of different types, specifically,vertical slurring generated by a film feeding operation and verticalslurring generated by a film winding operation.

As described above, when the vertical slurring effect is added, thenoise data (data where the acquired image data is not present) isdisplayed below the displayed image, which must be hidden. A hidingmethod will be described below. As for the captured image 1503 having novertical slurring, noise data is displayed on a lower part of the screenfor the captured images 1504 and 1505 where arbitrary vertical slurringhas occurred. As the hiding method, for example, a method for adding amask or a method for enlarging image data may be used.

In the method for adding the mask, for a captured image 1507, a maskimage 1506 where vertical slurring is equal to or larger than maximumslurring width (the offset amount) is superimposed on the lower part ofthe screen to hide the noise data. At this time, by superimposing a maskimage of an equal size in the upper part of the screen, the capturedimage 1505 having an aspect ratio of a letter box shape can bedisplayed.

On the other hand, the method for enlarging the clipped image data to anoriginal image size instead of adding a mask can also be employed. Inthis case, the system controller 114 enlarges the image data such that aheight of a region 1509 not including a maximum slurring width ofvertical slurring inside a captured image 1508 where vertical slurringoccurs, is equal to a screen height. Thus, the system controller 114displays the image data as displayed image 1510. At this time, an aspectratio of the screen must be maintained.

A clipping processing unit 129 reads, to provide a vertical slurringeffect, the captured image from the frame memory 115 at an arbitraryposition designated by a slurring amount calculation unit 128, andstores the captured image in the frame memory 115.

A mask generation unit 130 generates a mask image having a predeterminedvertical width. The predetermined vertical width can be set equal to ormore than a maximum slurring width added to current image data andacquired from the system controller 114, or a mask image storedbeforehand with a sufficient vertical width can be used. A masksynthesis processing unit 131 synthesizes a mask image 1308 generated bya mask processing unit 1307 with the captured image 1507 stored in theframe memory 115 at predetermined timing to output a synthesized image.

FIG. 16 illustrates correction of a luminance signal for realizingblinking according to the present exemplary embodiment. In the luminancecorrection processing, blinking of the input captured image is realizedby changing a luminance component of an output image based onsequentially changing input/output characteristics. A minimum unit of achange of the luminance signal is minimum resolution of the outputimage, and minimum time unit is an updating period of the capturedimage.

When correction is carried out in an order of characteristics 1601 as areference, and characteristics 1602, 1603, and 1604 as indicated bysolid lines, captured images are 1606, 1607, and 1608. In this case, abrightness order is an image 608>an image 1605>an image 1607>an image1606. To randomly generate blinking, for example, a plurality ofparameter data constituting input/output characteristics may beprepared, and the parameter data to be used may be determined based on arandom number. Various random numbers such as that of Gaussiandistribution can be employed, however, they are not limited to anyspecific types of random numbers.

The memory 135 illustrated in FIG. 1 stores a plurality of luminancecharacteristic data for determining input/output characteristics of theluminance of the video signal. For example, in the case wherecharacteristics 1601 to 1604 are represented by a linear function,parameter data includes inclination and intercept of a straight line. Apoint of clipping an output or a clipped value when an input is large,is also parameter data, as in the case of the characteristics 1604. Thedetermined parameter data is transmitted to a luminance correctionprocessing unit 127.

The luminance correction processing unit 127 corrects the input/outputcharacteristics of the luminance for the captured image read from theframe memory 115 according to the parameter data indicating thedetermined correction characteristics, and outputs the correctedcaptured image to the frame memory 115.

The system controller 114 designates a clipping position and a clippingsize from the grain noise memory 118, to the grain clipping processingunit 119, and designates a resizing amount to the grain resizingprocessing unit 120. The system controller 114 designates a synthesisratio of image data to be read and grain noise data 1203, to the grainsynthesis processing unit 121.

Similarly, the system controller 114 designates a clipping position anda clipping size from the scratch noise memory 121, to the scratchclipping processing unit 122 that synthesizes the scratch noise, anddesignates a resizing amount of the clipped scratch noise to the scratchresizing processing unit 124. The system controller 114 designates asynthesizing position of the scratch noise and synthesis ratio with thecaptured image data to the scratch synthesis processing unit 125.

Next, an operation including a user's operation for adding eachfilm-tone image effect according to the present exemplary embodimentwill be described.

The system controller 114 receives an operation of the key switch 103 asan input, and controls selection and determination of a film-tone imageeffect and controls the UI screen therefor. FIG. 17 illustrates the UIscreen for the film-tone image effect to be displayed on the displayunit 134. When the key switch 103 is operated to start setting of thefilm-tone image effect, the processing changes to a screen 1700.

A button 1701 is a button for ending the setting of the film-tone imageeffect on the UI screen. When the key switch 103 is operated, and thebutton 1701 is selected to execute the determination, the film-toneimage effect setting screen is ended, and an effect selected at thisstage is executed. A button 1702 on the UI screen is a button fordetermining enabling or disabling of a blinking effect and indicatingits state. A state of the button 1702 indicates that the blinking effectis disabled.

As for other film-tone image effects, namely, a distortion, grain noise,scratch noise, and vertical slurring, setting and states of the effectscan be grasped by a button similar to that of the blinking. The effectsetting buttons are controlled by a toggle operation. When the button1702 is operated in a disabled state of the blinking effect, an enabledstate indicated by a button 1704 is set. Conversely, when the button1704 is operated in an enabled state, a disabled state indicated by thebutton 1702 is set. Operations are similar for other effect buttons.

A screen 1703 displays a state where a distortion, vertical slurring,and blinking are selected among the film-tone image effects. A screen1705 displays a state where all the film-tone image effects areselected.

Next, referring to FIGS. 5, 6, and 9, a determination method of anexecution order of effects when a plurality of film-tone image effectsis selected, will be described in detail. The determination method ofthe execution order is characteristic processing of the presentexemplary embodiment. For example, it is assumed that the systemcontroller 114 has executed the film-tone image effects in order ofvertical slurring and scratch noise without taking an execution order ofthe effects into consideration.

In this case, since phenomena which bring about the effects are verticalslurring of an image being reproduced, caused by film travel during thereproduction, and the noise due to film surface damage caused by adevelopment work mistake, an original phenomenon order in filmphotographing is scratch noise and vertical slurring. When this order ischanged to the vertical slurring and the scratch noise, in an outputimage, the captured image vertically rises and falls while the scratchnoise does not move. As a result, the output image is different fromthat originally acquired in the film photographing.

Thus, according to the present exemplary embodiment, priority is givenbeforehand to each effect, and an execution order is determinedaccording to the priority. More specifically, as illustrated in FIG. 6,priority of the film-tone image effects is determined in an order ofphenomena which appear during the process starting from filmphotographing to reproduction. When a value of priority is larger, aneffect is executed earlier.

A table of FIG. 6 illustrates information about an image processingmethod for realizing each effect. An attribute is information indicatingcorrespondence between a phenomenon appearing during the filmphotographing, and each effect. According to the priority illustrated inFIG. 6, when all the effects are selected, the effects are executed inorder of a distortion (first processing), grain noise (secondprocessing), color fading (fifth processing), scratch noise (secondprocessing), vertical slurring (third processing), and blinking (fourthprocessing). The vertical slurring and the blinking both havereproduction attributes, and are phenomena appearing during thereproduction, and thus priority thereof is high. Priority between thevertical slurring and the blinking is in an order of the verticalslurring and the blinking since projection is carried out from the lightsource after film is fed, in an actual film reproduction process.

Next, referring to FIG. 9, the system controller 114 controls each unitto execute the plurality of film-tone image effects. In the presentexemplary embodiment, after a user's operation, the image effectprocessing unit 113 carries out each processing for image data acquiredfrom the image sensor 108 at a predetermined frame rate. However, thepresent invention is not limited to this. The processing can be appliedto moving image data stored in the recording medium 133 or input fromthe outside, or a plurality of image data continuously captured.

In step S901, it is evaluated whether execution of the film-tone imageeffects has been determined, from operation information of the keyswitch 103. When the execution has been determined (YES in step S901),the processing proceeds to step S902. When it is determined that theexecution has been cancelled, the processing proceeds to step S907.

In step S902, additional effect information indicating whether anyoperation of enabling or disabling an effect has been carried out isacquired for all the effects from a selection result on the UI screen.After information has been acquired, the processing proceeds to stepS903.

In step S903, the additional effect information acquired in the lastcontrol and the currently acquired additional effect information arecompared with each other to determine whether they are different. Whenthey are similar (NO in step S903), the control is ended determiningthat no film-tone image effect operation has been executed. On the otherhand, when there is a difference (YES in step S903), it is determinedthat a certain change has occurred with respect to adding any effect andthe processing proceeds to step S904.

In step S904, whether the additional effect information about adesignated effect acquired instep S902 is enabled is determined. Whenenabled (YES in step S904), the processing proceeds to step S905. Whendisabled (NO in step S904), the processing proceeds to step S906.

In step S905, as for enabled effects, values determined in priorityillustrated in the table of FIG. 6 are determined as priority of thedesignated effects. The higher numerical values, the higher thepriority. After the priority has been determined, the processingproceeds to step S907.

In step S906, as for disabled effects, priority of a designated effectis set to zero. Priority of a numeral zero is defined to be lower thanany other effects included in the table of FIG. 6. After the priorityhas been determined, the processing proceeds to step S907.

In step S907, whether priority has been determined for all the effectsis evaluated. When the priority has been determined for all the effects(YES in step S907), the processing proceeds to step S908. When not (NOin step S907), the effect determined in step S904 is then established,and the processing proceeds to step S909.

In step S908, an order of applying the effects is determined based onthe priority of each effect. Then, the processing proceeds to step S909.

In step S909, a setting parameter is determined for each image effect.At this time, as for the effect having its priority set to zero in stepS906, a parameter is set which instructs nonexecution of imageprocessing by inhibiting reading of an image from the frame memory 115,or causing the image to remain unchanged before and after the effect,thereby disabling the effect. After the setting parameter has beendetermined, the processing proceeds to step S911.

In step S910, a parameter is set which instructs nonexecution of imageprocessing for all the effects or causing the image to remain unchangedbefore and after the effects, thereby disabling the effects. After thesetting parameter has been determined, the processing proceeds to stepS911.

In step S911, the system controller 114 sets the setting parametersdetermined in steps S909 and S910 in each processing unit of the imageeffect processing unit 113, and causes each unit to process the imageeffect. The system controller 114 instructs execution of each processingin the order of the effects rearranged by priority in step S908.

The image effect processing unit 113 processes image data sequentiallyacquired at the respective processing units, and sequentially outputsthe image data to the next processing unit. When the processing enabledin the image effect processing unit 113 ends, the processed image datais sequentially output from the image effect processing unit 113. Aftersetting of the parameters and instruction of the execution order hasbeen completed, the control is ended.

In the case where the priority of the effects is set according to thefirst exemplary embodiment illustrated in FIG. 6, when all the effectsare enabled, image processing to be executed and, and an order ofexecution of the image processing for an input captured image 501 toacquire a final output image 508 are illustrated in FIG. 5. Images 502to 507 are stored in the frame memory 115. The effects of a distortion,grain noise, scratch noise, color fading, vertical slurring, andblinking have sequentially been applied to the images 502 to 507. Asclearly illustrated in FIG. 5, for example, the effects of verticalslurring and blinking are applied after the other image effects areapplied.

As described above, according to the first exemplary embodiment, whenthe plurality of types of film-tone image effects is applied to thecaptured image, priority is determined beforehand for each effect. Inthis case, among the film-tone image effects, the execution order of theeffects corresponding to the phenomena appearing during the reproductionare applied after the effects corresponding to the other phenomenaappearing during the photographing are applied as a priority order. As aresult, the phenomena appearing during the reproduction are preventedfrom occurring before the phenomena appearing during the photographing,and an image capturing result having effects closer to a film tone canbe acquired.

A second exemplary embodiment is directed to an image processingapparatus that can add a plurality of different effects as film-toneimage effects to an image. FIG. 2 is a block diagram illustrating adigital video camera as an image processing apparatus according to thesecond exemplary embodiment. The digital video camera includes: anoptical unit 200 that includes a lens, a diaphragm, and an actuator fordriving these components; an image capturing unit 201 that processessignals read from an image sensor to sequentially acquire and outputimage data; a recording unit 202 that records a video signal; and a keyswitch 203 that includes a group of switches for enabling the user toselect and determine film-tone image effects.

A group of lenses 204 and 205 forms an object image. A diaphragm 207adjusts the amount of incident light. A driving control unit 206includes an actuator for driving the lens 204 and the diaphragm 207 tohave instructed predetermined values. An image sensor 208photoelectrically converts incident light which passes through theoptical unit 200 to form an image. The photoelectrically convertedsignal is input to a camera signal processing unit 209. The camerasignal processing unit 209 performs various image processes on thephotoelectrically converted signal to convert it into a video signal.

An image effect processing unit 213 adds various film-tone image effectsto the video signal output from the camera signal processing unit 209.The video signal output from the image effect processing unit 213, onwhich a predetermined user interface screen (hereinafter, UI screen) issuperimposed, is displayed on a display unit 234. An encoder 232 codesthe video signal output from the image effect processing unit 213 in apredetermined recording format to write and store it in a recordingmedium 233.

Next, signal processing carried out by the camera signal processing unit209 will be described. The signal input to the camera signal processingunit 209 is separated into a luminance component and a color component.Various signal processes such as gamma processing, color matrixprocessing, and contour highlight processing are performed on the eachsignal component, namely, the separated luminance component and thecolor component. The processed signal is input as a captured image to aframe memory 215. The frame memory 215 stores the image processed by thecamera signal processing unit 209 and the image effect processing unit113. The image is written or read at predetermined timing.

The image effect processing unit 213 adds a plurality of differentfilm-tone image effects to the input image. The effects to be added are“distortion” for adding a distortion, “grain noise” for adding granularnoise, “scratch noise” for adding vertical noise, “color fading” forreducing saturation, “blinking” for adding fluctuation of a luminancelevel which changes with time, and “vertical slurring” for addingvertical slurring. The image effect processing unit 213 is divided intothree types of sub-blocks based on types of image processing for addingfilm-tone image effects.

A first sub-block is an image processing block for deforming image dataread from the fame memory 215. In the first sub-block, distortion andvertical slurring effects are realized. A second sub-block is an imageprocessing block for correcting luminance and color characteristics forthe image data read from the fame memory 215. In the second sub-block,color fading and blinking effects are realized. A third sub-block is animage processing block for synthesizing another image data with theimage data read from the fame memory 215. In the third sub-block, grainnoise and scratch noise effects are realized.

Next, image processing of a plurality of film-tone image effects by thesub-track and an execution order of processes in the sub-block will bedescribed in detail, which is characteristic processing of the presentexemplary embodiment. The first sub-block includes two image processingunits. Two image processing units add an effect of a distortion forreproducing a phenomenon caused by optical characteristics of the lensin a pseudo manner, in a process starting from film photographing toreproduction during the photographing, and an effect of verticalslurring for reproducing a phenomenon caused by film travel during thereproduction in a pseudo manner.

The first sub-block inputs a captured image stored in the frame memory215. A deformation amount calculation unit 216 calculates a deformationamount added to the input image. The deformation amount calculation unit216 determines a moving amount of an intersection point of each regionwhen the image input from the frame memory 215 is divided intopredetermined regions. A deformation processing unit 217 carries outgeometric deformation processing for the input image according to thedeformation amount determined by the deformation amount calculation unit216.

Further, the deformation amount calculation unit 216 determines variousmoving amounts based on parameters acquired from deformation amount data210 containing a plurality of deformation amount data parameters. Thus,an effect of an image where different distortion characteristics areadded to the same input image, in other words, an image captured by alens of optical characteristics of distortion types at a various levelor a distortion of a barrel type, and a bobbin type, can be provided.

A vertical slurring clipping processing unit 129 stores, to provide avertical slurring effect, the input image in a buffer memory (notillustrated). The image is read from the buffer memory at an arbitraryposition designated by a slurring amount calculation unit 228, andstored in the frame memory 215.

The second sub-block includes two image processing units that add aneffect of color fading for reproducing a phenomenon caused by a secularchange of film in a pseudo manner, in the process starting from filmphotographing to reproduction, and an effect of blinking for reproducinga phenomenon caused by a light source during the reproduction in apseudo manner.

The second sub-block inputs the captured image stored in the framememory 215. A color difference characteristic data storage unit stores aplurality of parameter data for determining color differencecharacteristics of a color difference of a video signal. For example, inthe case where characteristics 1401, 1402, and 1403 are represented by alinear function, parameter data includes inclination and intercept of astraight line.

The characteristics 1401 indicate normal characteristics showing nocolor fading. The characteristics 1402 indicate characteristics oflighter color than the characteristics 1401. The characteristics 1403indicate characteristics of much lighter colors than the characteristics1402. Correction characteristic data is determined according to a levelof a secular change, for example, the number of storage. The determinedparameter data is transmitted to a color difference correctionprocessing unit 226.

The color difference correction processing unit 226 corrects theinput/output characteristics of the color difference as for the capturedimage read from the frame memory 215 according to the parameter dataindicating the determined correction characteristics, and outputs thecorrected captured image to a luminance correction processing unit 227.The color difference correction processing is carried out afterprocessing of a camera signal processing unit 209. Thus, even when colordifference processing changes output characteristics, its result willnot affect color component processing executed by the camera signalprocessing unit 209.

A luminance characteristic data storage unit 212 stores a plurality ofparameter data for determining input/output characteristics of luminanceof the video signal. For example, in the case where characteristics 1601to 1604 are represented by a linear function, parameter data includesinclination and intercept of a straight line. A point of clipping anoutput or a clipped value when an input is large, as in the case of thecharacteristics 1604, is also parameter data. The determined parameterdata is transmitted to a luminance correction processing unit 227.

The luminance correction processing unit 227 corrects the input/outputcharacteristics of the luminance as for the captured image input fromthe color difference correction processing unit 226 according to theparameter data indicating the determined correction characteristics, andoutputs the corrected captured image to the frame memory 215. Theluminance correction processing is carried out after processing of thecamera signal processing unit 209. Thus, even when luminance correctionprocessing changes output characteristics, its result will not affectluminance component processing executed by the camera signal processingunit 209.

The third sub-block includes two image processing units. Two imageprocessing units add effects of grain noise and scratch noise forreproducing granular noise and linear noise caused by phenomenonprocessing in the process starting from the film photographing to thereproduction in a pseudo manner.

The second sub-block inputs the captured image stored in the framememory 215. A grain synthesis processing unit 221 reads the capturedimage stored in the frame memory 215, and synthesizes the image withgrain noise data 1203 at a predetermined synthesis ratio to store it inthe frame memory 215.

A scratch noise memory 222 stores scratch noise data 1301 including aplurality of patterns. The scratch noise data 1301 is read from thescratch noise memory 222. A scratch clipping processing unit 223 clipsscratch noise data 1302 at a designated position and of a designatedsize from the scratch noise data 1301. A scratch resiting processingunit 224 resizes the clipped noise data 1302 to scratch noise data 1304of a size required for synthesis with the captured image 103 stored inthe frame memory 215. A scratch synthesis processing unit 225 reads thecaptured image stored in the frame memory 215, and synthesizes the imagewith the resized noise data 1304 at a predetermined synthesis ratio tostore it in the frame memory 215.

A mask generation unit 230, which is not included in any of the first tothird sub-blocks, generates a mask image having a maximum slurring widthor more when a vertical slurring effect is selected. A mask synthesisprocessing unit 231 synthesizes, after the image processing has beencompleted in the first to third sub-blocks, a mask image 1308 generatedby a mask processing unit 1307 with the captured image 1507 stored inthe frame memory 115 at predetermined timing to output a synthesizedimage.

A system controller 214 controls the image sensor 208, the camera signalprocessing unit 209, and the image effect processing unit 213. Thesystem controller 114 designates a signal storage period or readingtiming to the image sensor 208. The system controller 114 sets, for thecamera image processing unit 209, parameters required for image qualitysetting in various signal processes. The system controller 114 acquiresevaluation values required for exposure control, focus control, andwhite balance control from the camera image processing unit 209.

The system controller 214 detects control positions of the lens 204 andthe diaphragm 207, determines, based on the evaluation values acquiredfrom the camera image processing unit 209, control values so that thecontrol positions can be placed at desired positions, and thendesignates the control values to the actuator 206.

The system controller 214 instructs each effect processing portion of afilm effect control unit 239 to determine an execution order of effects,and execute various setting and operations. The system controller 214designates a type and a level of a distortion to be added for thedeformation amount data 210 to acquire a deformation amount parameter.The system controller 214 sets a deformation amount to be set for thedeformation amount calculation unit 216.

The system controller 214 designates a size and a clipping position fromthe grain noise memory 218, to the grain clipping processing unit 219,and designates a resizing amount to the grain resizing processing unit220. The system controller 214 designates a synthesis ratio of acaptured image and grain noise data 1203 to the grain synthesisprocessing unit 221. Similarly, the system controller 214 designates asize and a clipping position from the scratch noise memory 221, aresizing amount of the clipped scratch noise, and a synthesizingposition of the scratch noise and a synthesis ratio with the capturedimage data, to the scratch clipping processing unit 222 that synthesizesthe scratch noise, the scratch resizing processing unit 224, and thescratch synthesis processing unit 225.

The system controller 214 designates a degree of a secular change forcolor difference characteristic data 211 stored in a memory 238illustrated in FIG. 2, and acquires color difference correction dataaccording to the degree. Then, the system controller 214 sets theacquired data in the color difference correction processing unit 226.

The system controller 214 instructs the slurring amount calculation unit228 to calculate a slurring amount, and designates a vertical slurringclipping processing unit 229 to execute a clipping operation. The systemcontroller 214 instructs the mask processing unit 237 to generate a maskimage.

The system controller 214 receives an operation of the key switch 203 asan input, and controls the display unit 214 to select and determine afilm-tone image effect and the UI screen therefor. FIG. 17 illustratesthe UI screen for the film-tone image effect displayed on the displayunit 234. When the key switch 203 is operated to start setting of thefilm-tone image effect, the processing changes to a screen 1700. Abutton 1701 is configured to end the setting of the film-tone imageeffect on the UI screen. When the key switch 203 is operated, and thebutton 1701 is selected to execute determination, the film-tone imageeffect setting screen is ended, and an effect selected at this stage isexecuted.

A button 1702 on the UI screen, is configured to determine enabling ordisabling of a blinking effect and indicate the determined state. Astate of the button 1702 shown in the drawing indicates that theblinking effect is disabled. Other film-tone image effects, namely, adistortion, grain noise, scratch noise, and vertical slurring can be setand grasped by buttons similar to that of the blinking.

The effect setting button is configured to carry out a toggle operation.If the button 1702 is operated when the blinking effect is in thedisabled state, an enabled state indicated by a button 1704 is set.Conversely, when the button 1704 is operated in the enabled state, andisabled state of the button 1702 is set. Similar operations areperformed on other effect buttons. A screen 1703 displays a state wherea distortion, vertical slurring, and blinking are selected among thefilm-tone image effects. A screen 1705 displays a state where all thefilm-tone image effects are selected.

Next, referring to FIGS. 7, 8A to 8D, and 10, a determination method ofan execution order of effects among the sub-blocks when a plurality offilm-tone image effects is selected will be described in detail. In thepresent exemplary embodiment, the film-tone image effects are executedby sub-block units. For example, when all the effects are selected to beexecuted, image processing is carried out in all of the first to thirdsub-blocks.

When the execution order of the effects among the sub-blocks is thefirst sub-block, the second sub-block, and the third sub-block, effectsof grain noise and scratch noise executed in the third block are lastprocesses to be performed. Vertical slurring or blinking is an effectappearing at a downstream reproduction stage among the process startingfrom film photographing to the reproduction. Accordingly, verticalslurring or blinking is added before granular noise or vertical noiseappearing at an upstream stage. In this case, a captures image whichvertically rises and falls while the scratch noise does not, is output,and a result is different from an effect originally acquired in the filmphotographing.

Thus, according to the present exemplary embodiment, priority is definedbeforehand for each effect, evaluation values are determined for eachsub-block according to priority, and an execution order is determinedaccording to the order of the evaluation values. FIGS. 8A to 8D aretables each illustrating a relationship in priority and evaluation valueamong various film-tone image effects. FIG. 8A illustrates evaluationvalues of the respective sub-blocks when all the effects are executed.Means described in the table of FIG. 8A indicates information about animage processing method for executing each effect.

An attribute is information indicating correspondence between aphenomenon appearing during the film photographing and a process stageof the phenomenon to which each effect corresponds to. A value ofpriority becomes larger toward the upstream stage of the process. Thehigher execution priority, the larger evaluation values. Thus, in anexample illustrated in FIG. 8A, the effects are executed in the order ofthe first to third sub-blocks.

Next, referring to FIG. 10, control where the system controller 214causes each unit to execute the plurality of film-tone image effectswill be described. In the present exemplary embodiment, after a user'soperation, the image effect processing unit 213 carries out eachprocessing for image data acquired at a predetermined frame rate fromthe image sensor 208. The present invention is not limited to this,however, this processing can be applied to moving image data stored in arecording medium 233 or input from the outside, or a plurality of imagedata continuously captured in time. In step S1001, whether execution ofthe film-tone image effects have been determined, is checked fromoperation information of the key switch 203. When the execution isdetermined (YES in step S1101), the processing proceeds to step S1002.When it is determined that the execution has been cancelled, theprocessing proceeds to step S1018.

In step S1002, additional effect information indicating whether anyoperation of enabling or disabling an effect has been carried out isacquired with respect to all the effects from a selection result on theUI screen. After information has been acquired, the processing proceedsto step S1003.

In step S1003, the additional effect information acquired in the lastcontrol and the currently acquired additional effect information arecompared with each other to determine whether they are similar ordifferent. When similar (NO in step S1003), the control is endeddetermining that no film-tone image effect operation has been executed.On the other hand, when there is a difference (YES instep S1003), theprocessing proceeds to step S1004 determining that a certain change hasoccurred in effect addition.

In step S1004, whether the additional effect information for adesignated effect acquired in step S1002 is enabled or disabled, isdetermined. When enabled (YES in step S1004), the processing proceeds tostep S1005. When disabled (NO in step S1004), the processing proceeds tostep S1006.

In step S1005, as for enabled effects, priority values determined in thetable of FIG. 8A are determined as priority of the designated effects.After the priority has been determined, the processing proceeds to stepS1007.

In step S1006, as for disabled effects, priority of a designated effectis set to zero. The numeral zero defines priority of an effect to belower than any other effects. After the priority has been determined,the processing proceeds to step S1007.

In step S1007, whether priority has been determined for all the effectsis checked. When the priority has been determined (YES in step S1007),the processing proceeds to step S1008. When not (NO in step S1007), theeffect to be checked in next step S1004 is determined, and theprocessing proceeds to step S1004.

In step S1008, priority of the effects in each sub-block is comparedwith one another, and priority of an effect having high priority, inother words, first in execution order, is acquired. In a configurationillustrated in FIG. 8A, priority of a distortion effect is acquired inthe first sub-block, priority of a color fading effect is acquired inthe second sub-block, and priority of a grain noise effect is acquiredin the third sub-block. After each priority has been acquired, theprocessing proceeds to step S1009.

In step S1009, values of the priority for the sub-blocks acquired instep S1008 are rearranged in descending order. In the configurationillustrated in FIG. 8A, a rearranged priority order is 4, 3, and 2.After the rearrangement, the processing proceeds to step S1010.

In step S1010, evaluation values of corresponding sub-blocks aredetermined according to the priority rearranged in step S1009.Evaluation values are 2, 1, and zero in descending order. In the exampleillustrated in FIG. 8A, evaluation values are 2 in the first sub-block,1 in the second sub-block, and zero in the third sub-block. Whenpriority is equal, and higher than the remaining effects, evaluationvalues are 2. Conversely, when priority is lower, evaluation values arezero. After the evaluation values have been determined for thesub-blocks, the processing proceeds to step S1011.

In step S1011, whether the evaluation values determined in step S1010are all different among the sub-blocks. In the example illustrated inFIG. 8A, the evaluation values of the sub-blocks are all different fromone another. Each of FIGS. 8B to 8C illustrates an example of aconfiguration of sub-blocks where among a plurality of effectsconstituting the sub-blocks, evaluation values of first effects areequal. In FIGS. 8B and 8C, priority is equal between the first andsecond blocks. In FIG. 8D, priority is equal among all the sub-blocks.When the evaluation values are all different, the processing proceeds tostep S1017. As illustrated in FIGS. 8B to 8D, when the evaluation valuesare equal between any given sub-blocks, the processing proceeds to stepS1012.

In step S1012, it is determined whether among the effects included inthe sub-block, an effect second in priority is enabled or disabled basedon the additional effect information acquired in step S1002. In theexample illustrated in FIG. 8B, the effects second in priority in thesub-blocks are all enabled. In the examples illustrated in FIGS. 8C and8D, the effects second in priority in the first and third sub-blocks areenabled, while the effect second in priority in the second sub-block isdisabled. When it is determined that the effect is enabled (YES in stepS1012), the processing proceeds to step S1013. When it is determinedthat the effect is disabled (NO in step S1012), the processing proceedsto step S1014.

In step S1013, priority of the second effect is acquired. In the exampleillustrated in FIG. 8B, priority of the first to third sub-blocks arerespectively 1, 2, and 1. After the priority has been determined, theprocessing proceeds to step S1015.

In step S1014, priority of a sub-block having no second effect isdetermined to be 5. This priority is larger in value than that of anyother effects. In the example illustrated in FIG. 8C, priority of thesub-blocks is respectively 1, 5, and 1. Similarly, in the exampleillustrated in FIG. 8D, priority of the sub-blocks is respectively 1, 5,and 2. After the priority has been determined, the processing proceedsto step S1015.

In step S1015, whether priority has been determined for all the effectsis determined. When priority has been determined (YES in step S1015),the processing proceeds to step S1016. When priority has not beendetermined (NO in step S1015), the processing proceeds to step S1012.

In step S1016, temporary evaluation values of the respective sub-blocksare determined according to the priority determined in steps S1013 andS1014. The temporary evaluation values are 2, 1, and zero in descendingorder. When priority is equal, and higher than other effects, temporaryevaluation values are 2. Conversely, when priority is lower, temporaryevaluation values are zero. In the example illustrated in FIG. 8B,temporary evaluation values are respectively zero, 2, and zero in thefirst to third sub-blocks. The acquired temporary evaluation values areadded to the evaluation values determined based on the priority of thefirst effects in step S1010 to be final evaluation values. In theexample illustrated in FIG. 8B, final evaluation values are respectively2, 4, and zero in the first to third sub-bocks. After the finalevaluation values have been determined with respect to the sub-blocks,the processing proceeds to step S1017.

In step S1017, the determined evaluation values of the respectivesub-blocks are rearranged in descending order. Image processing of therespective sub-blocks are executed in the rearranged order, andfilm-tone image effects are added. After the rearrangement, theprocessing proceeds to step S1018.

In step S1018, a setting parameter is determined for each image effect.At this time, for the effect having its priority set to zero in stepS106 or the effect having its priority set to 5 in step S1014, namely,as for an disabled effect, a parameter is set inhibiting reading of animage from the frame memory 215 Alternatively, a parameter is set whichmakes characteristics of the image similar before and after effect isadded to the read image, thereby disabling the effect. After the settingparameter has been determined, the processing proceeds to step S1020.

In step S1019, a parameter is set instructing not to execute imageprocessing for all the effects or a parameter is determined which makessimilar the characteristics of the read image before and after effect isadded to the read image, thereby disabling the effects. After thesetting parameter has been determined, the processing proceeds to stepS1020.

In step S1020, the system controller 214 sets the setting parametersdetermined in steps S1018 and S1019 in each processing portion of theimage effect processing unit 213. The system controller 114 instructsthe respective sub-blocks to execute the image processing in the orderof the effects rearranged in step S1017. After the instruction as tosetting of the parameters and the execution order has been completed,the control is ended.

FIG. 7 illustrates, according to the second exemplary embodiment in thecase of the priority of the effects illustrated in FIG. 8A, if all theeffects are enabled, what image processing is executed in what order foran input captured image 701 to acquire a final output image 708. Images702 to 707 stored in the frame memory 215 are images to which theeffects of a distortion, grain noise, scratch noise, color fading,vertical slurring, and blinking have sequentially been applied. Asillustrated, concerning the effects in the sub-blocks, the effect ofvertical slurring is carried out as a last effect of the firstsub-block, and the effect of blinking is carried out as a last effect ofthe third sub-block.

As described above, according to the second exemplary embodiment, whenthe plurality of types of film-tone image effects is applied to thecaptured image, the sub-block assembling a plurality of effects isconfigured. In this case, among the plurality of effects in thesub-block, the execution of the film-tone image effects corresponding tothe phenomena appearing during the reproduction such as verticalslurring and blinking are performed after the other effects areperformed, in priority order. As a result, the phenomena appearing inthe same sub-block during the reproduction is not applied before thephenomena appearing during the photographing at the upstream stage, anda result having effects closer to a film photographing can be acquired.

Further, according to the present exemplary embodiment, reading/writingof the image data between the frame memory 215 and the image effectprocessing unit 213 is carried out by sub-block unit. Accordingly, whenthe number of effects to be added increases, as compared with a case ofexecuting reading/writing from and to the frame memory 215, the numberof memory accessing times per unit time is reduced. This can increase amemory latitude of the frame memory 215. The reduced number ofreading/writing times reduces a delay of time from an input of thecaptured image into the image effect processing unit 213 to an output ofthe captured image. Thus, an image can be acquired with less delay timewhile adding the film-tone image effects.

In the example of the priority according to the present exemplaryembodiment illustrated in FIG. 8B, the first and second sub-blocks areequal in priority order as for the first effects. As a result, it is notappropriately determined which to execute first. However, by taking intoaccount the priority of the second effect in each sub-block, anexecution order can be determined to be the second sub-block and thenthe first sub-block. Accordingly, the second sub-block including manyupstream stages is executed first, and the first sub-block includingdownstream stages is then executed. Thus, according to the presentexemplary embodiment, by determining the execution order of thesub-blocks according to the types of the effects constituting thesub-blocks, a result having effects closer to a film photographing canbe acquired.

In the examples of the priority according to the present exemplaryembodiment illustrated in FIGS. 8C and 8D, only one of the effectsincluded in the second sub-block is enabled. In such a case, an enabledeffect of the second sub-block is set as a first effect irrespective oforder in the sub-block and as a priority, an execution order is placedbefore the other sub-blocks having pluralities of effects, so that thesecond sub-block is executed first. Thus, according to the presentexemplary embodiment, even when the user selectively determines effectson the UI screen, the execution order of the effects are appropriatelydetermined, and a result having effects closer to a film photographingcan be acquired.

The first and second exemplary embodiments have been directed to thecase of adding the plurality of film-tone image effects during themoving image capturing operation of the digital video camera. However,the present invention is not limited to the moving image capturing. Theinvention can be applied to a case of adding effects of filmphotographing in continuous capturing of a plurality of images orreproduction of a moving image.

The exemplary embodiments of the present invention have been describedin detail. However, the present invention is not limited to theexemplary embodiments. Various changes can be made without departingfrom the gist of the invention. Some portions of the exemplaryembodiments can appropriately be combined.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment (s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2012-006311 filed Jan. 16, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: an acquisition unit configured to sequentially acquire image data; a first processing unit configured to carry out image distortion processing on the image data; a second processing unit configured to add noise to the image data processed by the first processing unit; and a third processing unit configured to add slurring to the image data processed by the second processing unit and sequentially output.
 2. The image processing apparatus according to claim 1, further comprising a fourth processing unit configured to add luminance correction of luminance characteristics randomly determined in terms of time, to the image data processed by the third processing unit.
 3. The image processing apparatus according to claim 1, further comprising a fifth processing unit configured to correct a color of the image data output from the second processing unit, and output the processed image data to the third processing unit.
 4. A method for controlling an image processing apparatus, comprising: sequentially acquiring image data; executing first processing to distort an image periphery of the image data; executing second processing to add noise to the image data processed by the first processing; and executing third processing to add slurring to the image data processed by the second processing and sequentially output.
 5. A computer readable program describing a procedure of the control method of the image processing apparatus according to claim
 4. 6. A storage medium storing a computer readable program for causing a computer to execute each process of the control method of the image processing apparatus according to claim
 4. 